feat: Prepare Additional Conformity Test Scenarios (#3886) #3887
Conversation
…se validation (hiero-ledger#3886) Signed-off-by: Michał Walczak <[email protected]>
…h wildcard validation (hiero-ledger#3886) Signed-off-by: Michał Walczak <[email protected]>
…ith wildcard validation (hiero-ledger#3886) Signed-off-by: Michał Walczak <[email protected]>
Signed-off-by: Michał Walczak <[email protected]>
Signed-off-by: Michał Walczak <[email protected]>
…hance flexibility (hiero-ledger#3886) Signed-off-by: Michał Walczak <[email protected]>
…tilities (hiero-ledger#3886) Signed-off-by: Michał Walczak <[email protected]>
…maintainability (hiero-ledger#3886) Signed-off-by: Michał Walczak <[email protected]>
…iero-ledger#3886) Signed-off-by: Michał Walczak <[email protected]>
…nd wildcard validation (hiero-ledger#3886) Signed-off-by: Michał Walczak <[email protected]>
…ion (hiero-ledger#3886) Signed-off-by: Michał Walczak <[email protected]>
…orted method responses (hiero-ledger#3886) Signed-off-by: Michał Walczak <[email protected]>
…sults (hiero-ledger#3886) Signed-off-by: Michał Walczak <[email protected]>
…lob-related fields and improved wildcard handling (hiero-ledger#3886) Signed-off-by: Michał Walczak <[email protected]>
…er handling and added overrides for enhanced modularity (hiero-ledger#3886) Signed-off-by: Michał Walczak <[email protected]>
… expand transaction receipt test overrides (hiero-ledger#3886) Signed-off-by: Michał Walczak <[email protected]>
Signed-off-by: Michał Walczak <[email protected]>
… and modular improvements (hiero-ledger#3886) Signed-off-by: Michał Walczak <[email protected]>
Signed-off-by: Michał Walczak <[email protected]>
…reasons for unsupported EIPs (hiero-ledger#3886) Signed-off-by: Michał Walczak <[email protected]>
…plexTypes` logic for improved object/array handling (hiero-ledger#3886) Signed-off-by: Michał Walczak <[email protected]>
packages/server/tests/acceptance/data/conformity/utils/validations.ts
Outdated
Show resolved
Hide resolved
packages/server/tests/acceptance/data/conformity/utils/processors.ts
Outdated
Show resolved
Hide resolved
...ages/server/tests/acceptance/data/conformity/overwrites/eth_getTransactionCount/get-nonce.io
Show resolved
Hide resolved
|
@mwb-al thanks for taking care of this.
Given the size of this PR, I believe it would be better to mark them as skipped and fix them in a follow up PR. |
…iero-ledger#3886) Signed-off-by: Michał Walczak <[email protected]>
…to quality issues (hiero-ledger#3886) Signed-off-by: Michał Walczak <[email protected]>
…etup (hiero-ledger#3886) Signed-off-by: Michał Walczak <[email protected]>
Codecov ReportAll modified and coverable lines are covered by tests ✅ ❌ Your project check has failed because the head coverage (46.58%) is below the target coverage (80.00%). You can increase the head coverage or adjust the target coverage.
@@ Coverage Diff @@
## main #3887 +/- ##
===========================================
- Coverage 89.63% 46.58% -43.05%
===========================================
Files 87 81 -6
Lines 4989 4675 -314
Branches 1004 955 -49
===========================================
- Hits 4472 2178 -2294
- Misses 264 2178 +1914
- Partials 253 319 +66
Flags with carried forward coverage won't be shown. Click here to find out more. see 66 files with indirect coverage changes 🚀 New features to boost your workflow:
|
|
@acuarica I've commented out the other dependent tests for now, since using .skip caused Codacy to raise issues about high cyclomatic complexity and excessive method length. At the moment, Codacy is reporting decreased test coverage as the main issue. Is there a way to either temporarily lower the test coverage threshold, or mark the affected tests as ignored (e.g., in codecov.yml) while we’re in the process of updating the full set of conformity tests? This would let us merge updates step by step. |
|
Note that the codecov workflow is not required. You can still merge this even is the codecov workflow is failing. No need to change anything in the codecov config.
Aside note, it looks like codecov is not working properly here given the magnitude in the decrease of code coverage. We are taking at look at improving codecov workflow. |
Signed-off-by: Michał Walczak <[email protected]>
|
@mwb-al if all @konstantinabl's comments are addressed, feel free to resolve them so we can move forward. |
|
@acuarica |
|
Given it's not a priority, let's wait until Monday so we can have more eyes on this. So no need to Dismiss review. |
|
@mwb-al Why are we creating locally all the files from the execution apis, instead of fetching them as we used to? |
|
@konstantinabl |
I think fetching directly from the execution-apis is better in terms of staying in sync with what tests are in the repo there and in case any new endpoints popup and we miss them. But, since you already have approvals from my colleagues and except this i really like the rest of the work you have done with splitting the logic, I will approve! Thanks, great job |
|
Thanks for the approval! Just to clarify, I’m not copying all the files, only the ones I want to overwrite. const isCustom = fs.existsSync(path.join(overwritesDirectoryPath, directory, file));
const dir = isCustom ? overwritesDirectoryPath : directoryPath;If a file exists in the |
6 participants
Description:
This PR refactors the conformity test suite (
@conformity-batch-1) to improve code organization, readability, and maintainability. The main changes include:What Was Removed
What Was Added
utilsdirectory with modular utility functions:utils.ts- Core utility functionsconstants.ts- Test constants and configurationinterfaces.ts- Type definitions and interfacesoverwrites.ts- Override handling logicprocessors.ts- Test data processing functionsvalidations.ts- Validation utilitiestransactions.ts- Transaction-related utilitiesWhy These Changes Were Made
overwrites.ts)Validation Types in Override Files
The
/overwritesfolder contains various types of test validations:eth_blockNumber
eth_blockNumber/simple-test.io- Simple block number test with wildcard result validationeth_call
eth_call/call-callenv.io- Contract call with environment variables, wildcard resulteth_call/call-contract.io- Basic contract call testeth_call/call-revert-abi-error.io- Contract call with ABI error reverteth_call/call-revert-abi-panic.io- Contract call with ABI panic reverteth_call/call-callenv-options-eip1559.io- Contract call with EIP-1559 options, wildcard resulteth_chainId
eth_chainId/get-chain-id.io- Chain ID retrieval testeth_createAccessList
eth_createAccessList/create-al-contract.io- Create access list for contract interactioneth_createAccessList/create-al-value-transfer.io- Create access list for value transfereth_createAccessList/create-al-contract-eip1559.io- Create access list for contract with EIP-1559eth_estimateGas
eth_estimateGas/estimate-gas-contract.io- Gas estimation for contract callseth_estimateGas/estimate-gas-value-transfer.io- Gas estimation for value transferseth_estimateGas/estimate-gas-contract-eip1559.io- Gas estimation for EIP-1559 contractseth_feeHistory
eth_feeHistory/fee-history.io- Fee history with wildcards for gas ratios and base feeseth_getBalance
eth_getBalance/get-balance-blockhash.io- Balance at specific block hasheth_getBalance/get-balance.io- Standard balance retrievaleth_getBalance/get-balance-unknown-account.io- Balance for non-existent accounteth_getBlockByHash
eth_getBlockByHash/get-block-by-hash.io- Block retrieval by hasheth_getBlockByNumber
eth_getBlockByNumber/get-block-n.io- Block retrieval with multiple wildcard fieldseth_getBlockByNumber/get-finalized.io- Finalized block with hash and timestamp wildcardseth_getBlockByNumber/get-latest.io- Latest block with dynamic fieldseth_getBlockByNumber/get-genesis.io- Genesis block with dynamic fieldseth_getBlockByNumber/get-safe.io- Safe block with wildcard resulteth_getBlockTransactionCountByNumber
eth_getBlockTransactionCountByNumber/get-genesis.io- Genesis block transaction count with wildcardeth_getCode
eth_getCode/get-code-contract.io- Contract code retrievaleth_getCode/get-code-empty.io- Empty contract code testeth_getLogs
eth_getLogs/no-topics.io- Log query with invalid address parameter erroreth_getLogs/contract-addr.io- Log query with invalid topics parameter erroreth_getProof
eth_getProof/get-account-proof-blockhash.io- Account proof with unsupported method erroreth_getStorageAt
eth_getStorageAt/get-storage.io- Contract storage retrievaleth_getStorageAt/get-storage-invalid-key.io- Invalid storage key erroreth_getTransactionByBlockHashAndIndex
eth_getTransactionByBlockHashAndIndex/get-transaction-by-block-hash-and-index.io- Transaction by block hash and indexeth_getTransactionByBlockNumberAndIndex
eth_getTransactionByBlockNumberAndIndex/get-transaction-by-block-number-and-index.io- Transaction by block number and indexeth_getTransactionByHash
eth_getTransactionByHash/get-transaction-by-hash.io- Transaction retrieval by hasheth_getTransactionCount
eth_getTransactionCount/get-nonce.io- Account nonce retrievaleth_getTransactionReceipt
eth_getTransactionReceipt/get-access-list.io- Transaction receipt for access list transactionseth_getTransactionReceipt/get-blob-tx.io- Transaction receipt for blob transactionseth_getTransactionReceipt/get-dynamic-fee.io- Transaction receipt for dynamic fee transactionseth_getTransactionReceipt/get-legacy-contract.io- Transaction receipt for legacy contract transactionseth_getTransactionReceipt/get-legacy-input.io- Transaction receipt for legacy input transactionseth_getTransactionReceipt/get-legacy-receipt.io- Transaction receipt for legacy receipt transactionseth_sendRawTransaction
eth_sendRawTransaction/send-access-list-transaction.io- Send access list transactioneth_sendRawTransaction/send-blob-tx.io- Send blob transactioneth_sendRawTransaction/send-dynamic-fee-access-list-transaction.io- Send dynamic fee access list transactioneth_sendRawTransaction/send-dynamic-fee-transaction.io- Send dynamic fee transactioneth_sendRawTransaction/send-legacy-transaction.io- Send legacy transactionThe list includes all endpoints that required modifications - even if only due to the fact that the transactions from chain.rlp are not replayed before the tests start - in order for them to pass successfully.
Related issue(s):
Fixes #3886
Checklist